Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

The advent of transformer-based models such as BERT has led to the rise neural ranking models. These have improved effectiveness retrieval systems well beyond that lexical term matching BM25. While monolingual tasks benefited from large-scale training collections MS MARCO and advances in architectures, cross-language fallen behind these advancements. This paper introduces ColBERT-X, a generalization ColBERT multi-representation dense model uses XLM-RoBERTa (XLM-R) encoder support information (CLIR). ColBERT-X can be trained two ways. In zero-shot training, system is on English collection, relying XLM-R for mappings. translate-train, queries coupled with machine translations associated passages. Results ad hoc document several languages demonstrate substantial statistically significant improvements over traditional CLIR baselines.